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1.4.3 Bayes’ Rule 

Now we are ready to state one of the most useful results in conditional probability: Bayes' rule. Suppose that we know P{A\ B) , but we are 
interested in the probability P(B\A) . Using the definition of conditional probability, we have 


Dividing by P(A) , we obtain 


P(A\B)P(B) = P{AnB) = P(B\A)P(A). 

P(A\B)P(B) 


P(B\A) = 


P(A) 


which is the famous Bayes' rule. Often, in order to find P{A) in Bayes' formula we need to use the law of total probability, so sometimes 
Bayes' rule is stated as 


P(B 3 \A) = 


where B\ , B 2 , • • •, B n form a partition of the sample space. 


P(A\Bj)P(Bj) 
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Bayes' Rule 

• For any two events A and B, where P{A) ^ 0 , we have 

P(B\A) = 


P{A\B)P{B) 

P(A) ' 


If B\ , B 2 5 B% , • • • form a partition of the sample space S, and A is any event with P{A) ^ 0 , we have 

P{A\Bj)P(Bj) 


P(Bj\A) 


^PiAlB^PW 


Example 1.25 

In Example 1.24 . suppose we observe that the chosen marble is red. What is the probability that Bag 1 was chosen? 

• Solution 

o Here we know P[R | Bi ) but we are interested in PyB\ \ R) , so this is a scenario in which we can use Bayes' rule. We have 

P(R) 

0-75 x | 

0.6 

_ 5 _ 

12 ' 

P(R) was obtained using the law of total probability in Example 1.24, thus we did not have to recompute it here. Also, note that 
P{Bl\R) = . This makes sense intuitively because bag 1 is the bag with the highest number of red marbles. Thus if 

the chosen marble is red, it is more likely that bag 1 was chosen. 


P(B l \R) = 


Example 1.26 (False positive paradox [5]) 

A certain disease affects about 1 out of 10 , 000 people. There is a test to check whether the person has the disease. The test is quite accurate. 
In particular, we know that 
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• the probability that the test result is positive (suggesting the person has the disease), given that the person does not have the disease, is 
only 2 percent; 

• the probability that the test result is negative (suggesting the person does not have the disease), given that the person has the disease, is 
only 1 percent. 

A random person gets tested for the disease and the result comes back positive. What is the probability that the person has the disease? 

• Solution 

o Let D be the event that the person has the disease, and let T be the event that the test result is positive. We know 

P{D) = 1 


10 , 000 ’ 
P(T\D C ) = 0 . 02 , 
P{T C \D) = 0.01 

What we want to compute is P(D\T) . Again, we use Bayes' rule: 

P(T\D)P(D) 


P(D\T) = 


P(T\D)P(D)+P(T\D C )P(D C ) 
(1-0.01) xO.0001 

(1-0.01) xO.0001+0.02 x (1-0.0001) 

0.0049 


This means that there is less than half a percent chance that the person has the disease. 


Discussion: This might seem somewhat counterintuitive as we know the test is quite accurate. The point is that the disease is also very rare. 
Thus, there are two competing forces here, and since the rareness of the disease (1 out of 10,000) is stronger than the accuracy of the test (98 or 
99 percent), there is still good chance that the person does not have the disease. 

Another way to think about this problem is illustrated in the tree diagram in Figure 1.25. Suppose 1 million people get tested for the disease. 
Out of the one million people, about 100 of them have the disease, while the other 999 , 900 do not have the disease. Out of the 100 people 
who have the disease 100 X .99 = 99 people will have positive test results. However, out of the people who do not have the disease 
999,900 X .02 = 19998 people will have positive test results. Thus in total there are 19998 I 99 people with positive test results, and 
only 99 of them actually have the disease. Therefore, the probability that a person from the "positive test result" group actually have the 
disease is 


P(D\T) = 


99 

19998 + 99 


0.0049 
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